Lecture #11 


* Sorting Algorithms, part Il: 
— Quicksort 
— Mergesort 
° Trees 
— Introduction 
— Implementation & Basic Properties 
— Traversals: The Pre-order Traversal 
e On-your-own Study 
— Full binary trees 


But first... STL Challenge 


Give me a data structure that | can use to maintain 
a bunch of people’s names and for each person, 
allows me to easily get all of the streets they lived 
on. 


Assuming | have P total people and each person has 
lived on an average of E former streets... 


What is the Big-Oh cost of: 


A. Finding the names of all people who have lived on 
“Levering street”? 

B. Determining if “Bill” ever lived on “Westwood blvd”? 

C. Printing out every name along with each person’s 
street addresses, in alphabetical order (names and addresses 


in aloha-order). 


GOOD CODERS... 


OK, WE'VE CHANGED ">" TO 
">=". BUT THAT DOESN'T 
WORK EITHER. 

AND NOW? 
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— KNOW WHAT THEY'RE DOING 


9GAG 


Divide & Conquer Sorting Algorithms 
What’s the big picture? 


Quicksort and Mergesort are efficient 
“divide and conquer” sorting algorithms. 


They generally work as follows: 


. Divide the elements to be sorted 
Into two groups of roughly equal 
size. 

2. Sort each of these smaller groups 
of elements (conquer) using 
recursion. 

3. Combine the two sorted groups 


hese Gi one EE Se IABIN) steps 


Conquer Divide 


The Quicksort Algorithm 


. If the array contains only 0 or 1 element, return. 


. Select an arbitrary element P from the array 


(typically the first element in the array). 


. Move all elements that are less than or equal to P 


to the left of the array and all elements greater 
than P to the right (this is called partitioning). 


. Recursively repeat this process on the left sub- 


array and then the right sub-array. 


13 1 2120694077 


D Select an arbitrary item P from the a 


Move items smaller than or equal to 
larger items to the right; P goes in-be 


Recursively repeat this process on th 
Recursively repeat this process on th 


EE 
Sen 


SE 


beem AS 


Perey 


EE MBA History Bio ` Drop-out 


Major Major Major 


QuickSort 


Everything left of EE Major 
(our first P) is now sorted! 


QuickSort 


Finally, all items are sottégrything right 
of EE Major 


(our first P) is 
now sorted! 
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D&C Carta Nel #1 ~ 
Only bother ing | cifies the 
And here’s arrays of at least ng | df lement of 
to sort. 
DIVIDE d 
void Qui’ CONOLIER ment. t) 
i mE CONOUER eme left 
£ Apply our QS ns right ` 
int algorithm to the right 
3 half of the array. 
Pivotin artiLtion(Array,First, Last); 
QuickSo rray,First,PivotIndex-1); // left 


} 
} 


QuickSort(Array,PivotIndex+1,Last); // right 
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The QS Partition Function 


The Partition function uses the first item as the pivot 
value and moves less-than-or-equal items to the left 
and larger ones to the right. 


int Partition(int a[], int low, int high) And finally, return 
{ Së the pivot’s index 
int pi = low; p 
ine pivot = o ¿in the array (4) to 
do the QuickSort 
{ function. 
while ( low <= high && a[low] <= pivot ) 
Low++; 
while ( a[high] > pivot ) r 
high--; E 


if ( low < high ) r 
swap(a[low], a[high]); ? 


while ( low < high ); TT | 

swap(a[pi], a[high]); g S | 
pi = high; 

return(pi); 4 1121330524099 77354756 


0 1 2 3 4 5 6 7 8 9 10 11 
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Big-oh of Quicksort 


We first partition the E 
array, at a cost of n 
steps. 

Then we repeat the 
process for each half... 


We partition 7 Ta 


This is our pivot value. O 40 466977 
halves, eact re 

Our partition function 
steps, at at l | i 
steps. moves smaller values to n steps 

the left of the array, 
Then we ref) larger values to the right. 
process for A nalt.. A EN 40 46 17 
We partition each of the 4 So at each level, we don 
halves, each taking n/4 operations, and we have 
steps, at a total cost of n 
o. log,(n) levels, so we get: n 


log-(n). 
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| Quicksort - Is It Always Fast? 


Are there any kinds of input data where 
Quicksort is either more or less efficient? 


Yes! If our array is already sorted or mostly 
sorted, then quicksort becomes very slow! 


1 10 20 30 4050 60 70 


Let’s see why. 
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Worst-case Big-oh of Quicksort 


SE n steps 
We first partition the array, p 


at a cost of n steps. 


1 1020304050 60/70 
Then we repeat the process 


for the Téti&right groups... n-1 steps 


Ok, let's parties Qt e 
group then, | This is our pivot vdiue]10 20 30 40 5060 704 © 


Our partition func the pa “ition algorithm 
Then we rep moves smaller values tm nv val 
for the ees the left of the arr 


lar valiac tnt | e E 
This is our p/ -20 30 40 50 60 70'e 
‘ Wait our right group yait, there is ` "Hu" ati 
ve any values 
STILL has nearly n up to the left |}, Au the bigger 
items! pivot value! ‘already the 


, (It has n-2 items) right! 
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Worst-case Big-oh of Quicksort 


SE n steps 
We first partition the array, p 


at a cost of n steps. | 


Ge 1 0 20 30 40 50 60|70 


Then we Was ou 


for the sothe p; But wait, there is 
didn’t r no group to the left 
Ok, let’s) tothe le of the pivot value! 
group " We, 
right side! 
Then we repeat the process 
for the lettSyight groups... 


n-1 steps 


20 30 40 50 60 70 


Wait our right group 
STILL has nearly n 
items! 


(It has n-3 items) 
Seow array, © 30.40 50 60 70 
\ larger values to the right. 


Worst-case Big-oh of Quicksort 


What you'll notice is that 
each time we partition, we 
remove only one item off 
the left side! 


And if we only remove 
one item off the 
left side each time... 


We're going to have to g 
through this partitioning 
process n times to process 
the entire array! 


And if the partition algorithm 
requires ~n steps at each 
And we gdavé&vels deep... 


Then our algorithm is O(n7)! 


n steps 

Ji 10 20 30 40 50 60|70 

n-1 steps 
10 20 30 40 50 60 70 
7 n-2 steps 

20 30 40 50 60/70 
4 — ea. 

j 40 50 60 70 


Other Quicksort Worst Cases? 


So, aS you can see, an array that’s mostly in 
order will require an average of N? steps! 


As you can probably guess, Quicksort also has 
the same problem with arrays that are in reverse 
order! 


So if you happen to know your data will be 
mostly sorted (or in reverse) order, avoid 
Quicksort! 


It's a DOG! 


QuickSort Questions 


Can QuickSort be applied easily to 
sort items within a linked list? 


Is QuickSort a “stable” sort? 


Does QuickSort use a fixed amount 
of RAM, or can it vary? 


Can QuickSort be parallelized across 
multiple cores? 


When might you use QuickSort? 
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Mergesort 


The Mergesort is another extremely efficient sort - yet 
it’s pretty easy to understand. 


But before we learn the Mergesort, we need to learn 
another algorithm called “merge”. 
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Mergesort 


The basic merge algorithm takes two-presorted arrays 
as inputs and outputs a combined, third sorted array. 


K coute variables il, i2 to zero 
ASSIS" phidre items to copy.. 

=< EE in both 

E keep Allil] to output array B and il+ 
eke the smallest of Re two books 

e, Adw TO the new shel 

| been MË eegen SABLE 


sare. moved _ runs out conv the entire 


‘Merge Algorithm in C+ 
+ 


void merge(int data[], int nl, int n2, 
int temp[]) 


d 
int 11=0, 12=0, k=0; 
=>int *Al = data, *A2 = data + nl: 


while (il < nl || 12 < n2) 


if (il n1) 
temp[k++] = A2[i2++]; 
else if (i2 n2) 
temp[k++] = Al[il++]; 
else if (data[il] <= A2[i2]) 
temp[k++] = Al[il++]; 
else 
temp[k++] = A2[i2++]; 
} 
=—>for (int i=0;i<nl+n2; i++) 
data[i] = temp[i]; 


temp 


data 
} 


Here’s the C++ version of 
our merge function! 


You pass in an input array 
called data and the sizes of 
the two parts of it to merge: 

nl and n2 


The last parameter, temp, is 
a temporary array of size 
nl+n2 that holds the merged 


results as we loop. 
Finally, we copy our merged 


results back to the data 


array. 
1 4 11132125 30 
Al A2 


Mergesort 


OK - so what's the full mergesort alogrithm: 


Mergesort function : 


1. If array has one element, then return (it’s sorted). 
2. Split up the array into two equal sections 

3. Recursively call Mergesort function on the left half 
4. Recursively call Mergesort function on the right half 
5. Merge the two halves using our merge function 


Ok, let’s see how to mergesort a shelf full of books! 
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—> 


K HB. If array has 1 item, then return 
2. Spl 2. Split array in two equal sections 


3. Ca 


44Ca 
s] e 


3. Call Mergesort on the left half 
4. Call Mergesort on the right half 
5. Merge the halves back together 


23 To save time, well skip 
the tracing on this 
side... 


. If array has 1 item, then return 

. Split array in two equal sections 
. Call Mergesort on the left half 

. Call Mergesort on the right half 
. Merge the halves back together 
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1. If array has 1 item, then return 
2. Split array in two equal sections 
3. Call Mergesort on the left half 
4. Call Mergesort on the right half 
. Merge the halves back together 


And our array is 
sorted!!!! 
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Mergesort - One Final Detail 


While | showed the Mergesort moving 
books into a bunch of small piles... 


Jata in- 


i or merging. 
al ‘ks! 


has one element, then return. 
array in two equal sections 
gesort on the left half 

gesort on the right half 

. Merge the two halves back together 


J rrervyge UT LNN KM TIUIVE D VACK tyygetirel 


Ther 


and only 
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e É «ý 4 
FES piy EN À 
Klak (AE big 1 


Big-oh of 
Mergesort 


log,n levels deep 


hy? Because we 
keep dividing our 
piles in half... 


until our piles are 
just 1 book! 


Big-oh of 
Mergesort 


log,n levels deep 
hy? Because we 

keep dividing our 
piles in half... 


until our piles are 
just 1 book! 


n items merged 
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n items merged 


gg éi 
n items merged 


Overall, this gives us n:log,(n) steps to 
sort n items of data. Not bad! [] 


Big-oh of 
Mergesort 


log,n levels deep 
hy? Because we 

keep dividing our 
piles in half... 


until our piles are 
just 1 book! 


Mergesort - Any Problem 
Cases 


So, are there any cases 
where mergesort is less 
efficient? 


No! Mergesort works 
equally well 
regardless of the 
ordering of the data... 


However, because the merge function needs 
secondary arrays to merge, this can slow things 


down a bit... 
In contrast, quicksort doesn’t need to allocate any 


new arrays to work. 
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MergeSort Questions 


Can MergeSort be applied easily to 
sort items within a linked list? 


Is MergeSort a “stable” sort? 


Are there any special uses for MergeSort 
that other sorts can’t handle? 


Can MergeSort be parallelized across 
multiple cores? 


i Sorting Overview 


Stable/ 
Non- 
stable 
Selection Unstabl Always O(n?), but simple to implement. Can be used with linked 
Sort e lists. Minimizes the number of item-swaps (important if swaps are 
slow) 
Insertion Stable O(n) for already or nearly-ordered arrays. O(n2) otherwise. 
Sort Can be used with linked lists. Easy to implement. 
Bubble Stable O(n) for already or nearly-ordered arrays (with a good 
Sort implementation). O(n?) otherwise. Can be used with linked 
lists. Easy to implement. Rarely a good answer on an 
interview! 
Shell Unstabl O(n*25) approx. OK for linked lists. Used in some embedded 
Sort e systems (eg, in a car) instead of quicksort due to fixed RAM 
usage. 
Quick Unstabl O(n log,n) average, O(n?) for already/mostly/reverse ordered 
Sort e arrays or arrays with the same value repeated many times. Can 


be used with linked lists. Can be parallelized across multiple 
cores. Can require up O(n) slots of extra RAM (for recursion) in the 
worst case, O(log,n) avg. 


Merge Stable O(n log,n) always. Used for sorting large amounts of data on 

Sort disk (aka “external sorting”). Can be used to sort linked lists. 
Can be parallelized across multiple cores. Downside: Requires 
n slots of extra memory/disk for merging - other sorts don’t 
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DOUNO AIOOTICNENS 
Visualized! 


Challenge Problems 


1. Give an algorithm to efficiently determine 
which element occurs the largest number of 
times in the array. 


2. What's the best algorithm to sort 1,000,000 
random numbers that are all between 1 and 
5? 


3. What are their Big-Os? 
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when your code is meant to be 
O(NIogN) but it's been 30 
minutes and it still hasn't finished 
N=3 


*breath in* 
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“| think that | shall never see a data structure as 
lovely as a tree.” - Carey Nachenberg 


Tree Data Structures 
What's the big picture? 


A tree is a data structure that stores 
values in a hierarchical fashion, e.g., 


We often use linked lists to build trees. For Uses: 
instance, the tree above has nodes with fficient searching, 
two “next” pointers - one going left and a 


one right. . i 

Trees are an alternative e linked lists and 
arrays when you need more organization of 
your data. 
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Trees 


A Tree is a special linked list-based data structure 
that has many uses in Computer Science: 


* To organize hierarchical data A Decision Tree 
* To make information easily AABI À pets spree 
searchable TE a fever? 
* To simplify the evaluation of yes/ D S 
mathematical expressions ES ère 
° To make decisions have spots on. havea sore 
his/her face? throat? 
A A A 
He has (Cult lt ar 


COOTIES! NULENULE 
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Basic Tree Facts 


1. Trees are made of nodes 
(just like linked list nodes). 


Every tree has a "root" pointer. 


Empty ree NULL 


root ptr 


. The top node of.a tree ~ 
is called its But instead of just one next 
ointer, a tree node can  dren 
4. Every dc GE two or more next 
or more pointers! p 
5. A node wit n is 
called a “leaf 4 -3 1 child 


fetes e root 
pointer is like a 
linked list’s hea 
pointer! 


some value 


ig Leaf node 


node *rootPtr; 


53 91 


-115 
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Tree Nodes Can Have Many Children 


A tree node can have more than just two children: 


struct node 
{ 
int value; // node data 

de *pChild1, *pChild2, *pChild3, …; 

ne kale et PS root ptr 
struct node 
{ 

int value; // node data 3 


node *pChildren[26]; 


}; 


7 4 


mes A 
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Binary Trees 


A binary tree is a special form of tree. In a binary 
tree, every node has at most two children nodes: 


A left child and a right child. 


struct BTNODE // binary tree node | 
{ A Binary Tree 
string value; // node data “carey” 
BTNODE *pLeft, *pRight; 
}; 
“leon” “andrea” 


“sheila” simon’martha™ milton 
NULANUL NUL ENUL | 


Binary Tree Subtrees 


We can pick any node in the tree... 


And then focus on its “subtree” - which includes it 
and all of nodes below it. 


/ This subtree includes four  \ 
different nodes... 


It has the “leon” node 


ANS as its root. d 
like this “leon” 


ON 


"sheila”fsimon’? 
NULI 


ziggy” 
NULANUL. 


Binary Tree Subtrees 


If we pick a node from our tree... 
we can also identify /ts left and right sub-trees. 


like this 


milton” 


“sheila!” Een 


— 


Operations on Binary Trees 


The following are common operations that we 
might perform on a Binary Tree: 


enumerating all the items 

searching for an item 

adding a new item at a certain position on the tree 
deleting an item 

deleting the entire tree (destruction) 

removing a whole section of a tree (called pruning) 
adding a whole section to a tree (called grafting) 


We'll learn about many of these operations over 
the next two classes. 
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struct BTNODE // node A Simple Tree 
{ 

int value; // data 

BTNODE *left, *right; Fxamnle ~ 
}; 

OS - cal As with linked lists, we 
Hs racarua © use dynamic memory to 
L> BTNODE *temd O2 allocate our nodes. ) 
—+ pRoot = new LUUU 
OR Al of memory for value, 5 

me? i 
—+ temp = new B 
— temp->value OS - can you 
— temp->left = reserve 12 bytes 
—>temp->right 1200 
—> pRoot->lef of memory for 7 
me? temp-> 00 
am value -3 

temp = new B j S 
—>temp->value = -3; NULL NU ure, left right 
—temp->left = NULL; chunk Oo Nu NULU/ 
—>temp->right = NULL; at address LU. 
—pRoot->right = temp; And of course, later we’d 

// etc... 


have to delete our tree*s—_ 


We've created a binary tree... 
now what? 


Now that we've created 
a binary tree, what can 
we do with it? 


Well, next class we'll learn 
how to use the binary tree 
to speed up searching for 


data. 
But for now, let’s learn how 


to iterate through each 
item in a tree, one ata 


time. SCH a on 
This is called “traversing 


the tree, and there are 
several ways to do it. 
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Binary Tree Traversals 


Just as we can traverse through linked lists, 
we can also traverse through the nodes of a tree. 


All tree traversals start 
with the root node. 


root 


WEN 


HEL 


There are four common ways to traverse a tree - 
and each visits the tree's nodes in a different order: 


nearly identical algorithms. 


Pre-order In-order Post-order 


Traversal Traversal Traversal 


These traversals all use recursion, and are 


Level-order 
Traversal 


L.O.T. uses a 
breadth-first-search 
to visit nodes. 


47 rre-oraer, In-oraer and rost- 
order 


Let's see the code for simple pre-, in-, and post-order 
traversals! 


void preorder(Node 
{ 


if (p = n 


Then we "process" 
the current node's 


Finally, we fully process 


the right subtree of the light); de value. 
current node using p bell left subtree of the current 
recursion. node using recursion. 


< 


x 


# 2 #3 


48 rre-oraer, In-oraer and rost- 
order 


Let's see the code for simple pre-, in-, and post-order 
traversals! 


void preorder(Node 
{ 
if (p == nullptr) 
return; 
cout << p->value; 
preorder(p->left) ; 
preorder (p- 
>right); 


"e 
nat 


S 


Are 
E ge ae: 


49 rre-oraer, 


In-oraer and FOSt- 


Let's see the code for simple pre-, in-, and post-order 


void preorder(Node 
{ 
if (p == nullptr) 


return; 
cout << p->value; 


preorder(p->Left) ; 


order 
traversals! 
void inorder(Node 
{ 


if (p == nullptr) 
=> return; 
cout << p->value; 

inorder(p- 
>right); 


right subtrees. 


In pre-order, we process pr 
the current node first, 
then process its left and 


—> inorder(p->leftk= 


ru i subtrees. 


g 
Next let's see the in- 
| order traversal! 


How's it different 


in between processing 
its left and right 


50 rre-oraer, In-oraer and FOSt- 


order 


Let's see the code for simple pre-, in-, and post-order 


traversals! 


First we fully process the 
left subtree of the current 
node using recursion. 


void postorder (Node 


{ 
if (p == nullptr) 
=> return; 


en we fully process the Elek postorder(p- 


right subtree of the 
Sy node using 
ecursion. d 


TI LHIIL JI SCHER 


# 2 
| 


teft); 


ELS. x 

Finally, after 
everything else, we 

"post-process" the 


post-order traversal! 


51 rre-oraer, In-oraer and FOSt- 


order 
Let's see the code for simple pre-, in-, and post-order 
traversals! 
void preorder(Node void inorder(Node void postorder(Node 
*p) *p) *p) 
{ 
if (p == nullptr) if (p == nullptr) if (p == nullptr) 
return; return; return; 
cout << p->value; inorder(p->Lleft)|; postorder(p- 
preorder(p->left); cout << p->value; | >left); | 
preorder(p- inorder(p- postorder(p- 
>right); >right); >right); 
cout << p->value; 


} 
Notice that the only difference is placement of the processing 


step! 
But that small change results in major differences in the order 
the nodes are processed! 


Ok, let's trace through our pre-order traversal! 
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The Pre-order Traversal 
Output: 


cur—nullptr 


—woid preorder(Node *cur) 
x = sa DI 
On d if (cur == nullptr) Otherwise we “process 
{4 return: the value in the current [0 
" | 
We node... Ie: À 
S preorder(c eft); // Process nodes in left sub-tree. ge *root: 
preorder(cur->right); // Process nodes in right sub-tree. 
11} 
} } Lreorder(root) 
} J 
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The Pre-order Traversal 
Output: 
USCkids have 


cur—nullptr 


void preorder(Node *cur) 
d 
i { if (cur == nullptr) // if empty, return... 
| return; ( 
cout << cur->value; // Process the current node. 
preorder(cur->left);  // Process nodes in left sub-tree. de "root: 
|| preorder(cur->right); A Process nodes in right sub-tree. 
He aa 
} à preorder(root); 
} I 


54 
The Pre-order Traversal 


Output: 
USCkids have 


void preorder(Node *cur) 


v{ 
\{  if(cur == nullptr) // if empty, return... 
| return; in() 
cout << cur->value; // Process the current node. 
ode *root; 


preorder(cur->left);  // Process nodes in left sub-tree. 
1 =>preorder(cur->right); // Process nodes in right sub-tree. . 
OZ preorder(root); 


5 


5 


Output: 


USCkids have 


The Pre-order Traversal 


J + Cur 
—wvoid preorder(Node *cur) 
v { 
\ { if (cur == nullptr) // if empty, return... 
return, n() 
cout << cur->value; // Process the current node. 
ode *root; 
—>preorder(cur->left); // Process nodes in left sub-tree. 
= — preorder(cur->right); // Process nodes in right sub-tree. 
+ 
} preorder(root); 
} J 
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The Pre-order Traversal 


Output: 
USCkids haveno 


CUT — "kids" 


"have' 
NULINUL I 


"no" 
INULINULL 


void preorder(Node *cur) 


\{ 


if (Cur == nullptr) 
return; 


cout << cur->value:; 


preorder(cur->left); 


=} 


// if empty, return... 


// Process the current node. 


// Process nodes in left sub-tree. 


+ —~preorder(cur->right); // Process nodes in right sub-tree. 


ain() 


Node *root; 


preorder(root); 


} 


i 
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The Pre-order Traversal 
Output: 
USCkids haveno 


Cur 


—woid preorder(Node *cur) 


V { 
{ — if (cur == nullptr) // if empty, return... 
return; 
cout << cur->value; // Process the current node. 


—preorder(cur->left); // Process nodes in left sub-tree. 
= —preorder(cur->right); // Process nodes in right sub-tree. 


= 


ain() 


Node *root; 


preorder(root); 


i Si 


